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ABSTRACT 

This study is a research project focused on the implementation of problem-based learning in an honors, paleontology-oriented, 
Earth Science course. The course, the Age of Dinosaurs, is taught at the University of Texas at Dallas to undergraduates from a range 
of majors who seek core-curriculum science credit. All class work is centered on fossils and rock samples selected from a research 
collection accumulated from dinosaur-bearing beds in the Chihuahuan Desert of West Texas. A list of goals for the course was 
prepared during the design phase for the class. This list was used as a Likert scale questionnaire following completion of the course 
and dissemination of grades. Results from the goals questionnaire and from the official course evaluations present highly favorable 
responses to the course and to the problem-based learning methodology. Students provided free-response assessments and advice 
that influenced the next offering of this course. © 2014 National Association of Geoscience Teachers. [DOI: 10.5408/13-085.1] 
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INTRODUCTION 

Scientists are natural, problem-based learners, intuitive¬ 
ly functioning within a problem-based learning (PBL) 
process. They continually contemplate poorly defined, 
open-ended questions that must be answered to specific 
standards. They frequently establish associations with 
colleagues to gather and consider diverse ideas. These 
researchers set expectations, perform authentic (real-world) 
experiments, maintain rigor, evaluate findings, write up 
results, create presentations, and present final products. We 
would argue that this process is not the typical expectation 
for students in the usual college science class. 

During the fall semester of 2012, the class of 20 students in 
the Collegium V (CV) Honors program met for a 75-min class 
twice each week. The philosophy of the CV program at the 
University of Texas at Dallas is as follows (Collegium V, 2013): 

• To respond to the academic needs of our most 
outstanding and academically ambitious students 

• To offer creative, challenging honors instruction that 
enriches the learning experience for both students and 
faculty by emphasizing interdisciplinarity 

• To promote and facilitate closer, more productive 
interaction among students and faculty 

• To nurture the highest level and quality of intellectual 
conversation on campus 

• To promote undergraduate scholarship and research 
across all disciplines of the university 

COURSE DESCRIPTION: AGE OF 
DINOSAURS 

Age of Dinosaurs (AoD) at the University of Texas at 
Dallas is an introductory course that focuses on student 
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engagement with the anatomy, physiology, ecology, and 
evolution of dinosaurs and aquatic and aerial reptiles, 
invertebrates, plants, and ichnofossils, as well as Mesozoic 
climates and basic Earth history of the age of dinosaurs. 
Introductory physical geology topics are addressed, as 
students are responsible for identifying enclosing rocks and 
for producing reasonable, but not particularly sophisticated, 
sedimentological interpretations. Each class provides an 
encounter with Mesozoic fossils and rocks from the research 
collections of Montgomery and colleagues at excavations 
near Terlingua in the Big Bend region of Texas. Approxi¬ 
mately 90 fossils and 10 rock specimens are the basis for 
class exploration, discovery, and study. 

AoD was taught during the fall semester of 2012 and 
was taught again in the spring of 2014. The instructors' 
experiences with PBL within the UTeach program (Mont¬ 
gomery is codirector of UTeach Dallas, and Donaldson is a 
UTeach Dallas master teacher) provided both the effective 
methodology and extensive experience that are necessary to 
achieve better comprehension and to meet problem-solving 
goals than had been the case with traditional lecture/ 
laboratory classes. 

AoD is an experiment into best educational practice. The 
curriculum is fundamentally a set of problems designed to 
address components that are typical of courses in paleon¬ 
tology, Earth history, and physical geology. The AoD process 
involves becoming proficient at problem solving while 
practicing self-directed learning in a group setting, a process 
described by Barrows and Kelson (1993). Learning is clearly 
oriented toward depth of understanding rather than simple 
breadth of knowledge. Much of the theoretical underpinning 
for educational practices in AoD was developed in Mont¬ 
gomery's Basis of Evolution class and in Donaldson's PBL 
course. (The AoD experience mostly substitutes fossils from 
which concepts must be derived for the theoretical examples 
in Basis of Evolution. It also draws from teaching practitioner 
training, which is an important component in PBL.) 

PBL methodology offers an innovative approach to 
teaching Earth Science for mostly nonmajors—in this case, 
to honors undergraduates. Specimen identification and 
interpretation within each group followed by presentation 
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of results and formal write-ups were the standard weekly 
activities. Questionnaire results support our claim that this 
course approximates the field/laboratory experience of some 
Earth scientists, particularly field-oriented paleoecologists 
and paleontologists. 

RESEARCH OBJECTIVE 

Our research objective was to determine the effective¬ 
ness of this AoD course in producing a practical, integrated 
framework in which students have meaningful learning 
experiences and achieve expected learning outcomes. 

PBLAPPROACH 

The PBL approach to education is a constructivist 
methodology that is recognized for its successes in 
promoting comprehension and developing problem-solving 
skills in a range of subjects across the education spectrum 
(Boss, 2011). We found such successes increasingly difficult 
to achieve in our undergraduate course offerings, even with 
honors students. The time for a methodology rethink was at 
hand. PBL methodology is not widespread in undergraduate 
science courses (for an overview, see Charlton-Perez, 2013). 
The AoD course is our first experiment at the University of 
Texas at Dallas in producing a core-curriculum, PBL science 
offering for mostly non-Earth Science majors. 

PBL is differentiated from project-based learning by a 
consideration of initial instructions. In PBL, a problem is 
outlined that must be addressed. In AoD, such a problem is 
put forward. Students work in groups over a period of time 
to provide solutions to this problem. In project-based 
learning, a specific problem may or may not be addressed. 
Otherwise, much of the same methodology as described by 
Markham et al. (2003) is employed. 

COURSE DESIGN 

Expected Learning Outcomes 

Course objectives for AoD as stated in the syllabus are as 
follows: 

1. Students will be familiar with evolutionary history of 
numerous Mesozoic fossils. 

2. Students will have a basic understanding of Mesozoic 
geology and paleoecology. 

3. Students will evaluate fossils from several of our 
dinosaur sites in west Texas using criteria presented 
in the class and in primary sources. 

4. Students will apply their learning and understanding 
to real-world research projects. 

5. Students will produce an original research project of 
extraordinary quality. 

This course is fundamentally concerned with promoting 
and developing problem-solving skills both in the individual 
student and in students functioning in groups. The intent of 
this course is not for students to gain the breadth of 
experience with diverse fossils and rocks that can be attained 
in excellent, undergraduate classes or an understanding of all 
of the numerous concepts required in physical geology 
courses. Students did, however, have meaningful, research- 
driven experiences with fundamental concepts that were 


developed within a practical, integrated framework. We 
would also argue that a semester of intensive research 
produced students who have great depth of knowledge and, 
as the occasion demands, who will efficiently seek answers 
from reliable sources to widen their depth of understanding. 

The course promoted expertise in collaborative learning. 
Much of the value of collaborative learning is the required 
use of oral language to put thinking into words. This process 
refines the student's thinking, which becomes more clarified, 
and the cognitive load for the writing/presenting component 
is lightened. The progression we seek to exercise and to 
make more efficient is a recursive process among thinking, 
language, and writing. 

No attempt was made to test the students with 
questions from standard Earth Science courses for two 
reasons. First, we were less interested in whether the 
students gained a breadth of specific knowledge than we 
were focused on problem solving. It was clear that when the 
members of one group excelled in stratigraphic minutiae, 
students in another group became experts in anatomy. Thus, 
expertise was highly diverse within an overall framework of 
Earth history, paleontology, paleoecology, and so forth. 
Using group work, paired with a presentation format, 
allowed all students access to the specific knowledge 
constructed by each team. Granting students a significant 
measure of control (choice) for what they are learning has 
been shown to promote self-confidence and responsibility 
(Glynn and Koballa, 2006). Postcollaboration presentation 
discussions and questions deepened and broadened their 
learning. 

Second, we were interested in whether an experiment, 
wherein students must operate within an authentic research 
realm, would be seen as successful by both students and 
faculty. One semester's course could be compared to the 
next in a qualitative way, but an attempt at quantitative 
comparison might prove difficult to interpret because of the 
shifting composition of the specimen collections and the 
somewhat unpredictable choices by the students for the 
details that they choose to study and the questions they 
choose to answer. 

Class Structure 

The immediate task during the first class of each week is 
for students to determine what has been discovered. 
Students were presented with a box or bag containing 
fossils ostensibly recovered during the past week at the site. 
By the end of this first class, students had identified the 
fossils (sometimes with a good bit of instructor assistance) 
and were beginning to discern how the fossils fit into the 
jigsaw of Mesozoic life and Earth history. We felt confident 
that once students at least had the identifications correct, 
they could effectively pursue their research. Based on 
conversations during the semester, we estimated that 
students spent at least 2 h each week seeking online, peer- 
reviewed references. Assembling the weekly analysis and 
updating the overall reconstruction required an additional 2 
to 4 h of work, much of it conducted in collaboration with 
other group members. 

As this recursive process continues and as the students 
near the end of the course, a dynamic and detailed 
paleontological and paleoecological reconstruction emerges. 
All students engage in probing discussions, activities, and 
presentations as they encounter numerous controversial 
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topics such as behavior, biomechanics, homeothermy, soft 
tissue preservation, origin of birds, and extinction and as 
they complete weekly assignments about their discoveries. 
Final presentations consist of detailed, research-based 
interpretations using mostly fossil data to reconstruct the 
AoD in West Texas. The production of highly creative 
presentations is encouraged. 

As is common in PBL classes, lectures are limited and 
are driven by student needs (Fyrenius et al., 2005). In AoD, 
lectures now persist no longer than a few minutes. These 
minilessons occur mostly when more than a couple of 
students require clarification or direction. Demonstrations, 
hands-on activities, short video clips, and whatever explan¬ 
atory methods emerge as needed accompany these brief 
lessons. These teachable moments are referred to as 
workshops, and they combine with student research and 
group discussion to replace lecture as the main means of 
gaining information in the class. Early in the semester, a 
discussion about effective research strategies is a priority. 
Students conduct a great deal of research both in and out of 
class. 

The open framework and self-directed nature of the 
course might be expected to lead to opportunities for less 
motivated students to participate in only a peripheral way. 
This has proven not to be the case for our honors courses, 
but preliminary and limited experiments with PBL in 
nonhonors courses suggest that effective measures must be 
devised for some of the less-motivated students. Each 
student has a responsibility to attend class and to be 
prepared. Course requirements are mostly self-policing, as a 
flurry of new specimens appears each week and are only 
available in class. Our impression, based on personal 
observations and on interviews with students, is that peer 
pressure and peer evaluations (which can influence a 
student's grade) that are carried out within a group contract 
framework are also effective motivators. 

PBL Elements 

PBL methodology is variable depending on the literature 
consulted, but for the purpose in this class, the Project-Based. 
Learning Handbook (Markham et al., 2003) served as a guide. 
Modifications are derived from several years of collective 
experience with PBL methodology. The following sections 
describe major components one usually finds in a PBL 
course such as this. 

Anchor Video and Entry Event 

The first class of the semester begins with an anchor 
video to engage students in believable scenarios that present 
complex problem-solving opportunities (Kumar, 2013). The 
key is to elicit engagement rather than apprehension at the 
prospect of studying so much unfamiliar material. In the 
video, students learn that the Big Bend of Texas is an 
extraordinary window into life at the end of the Cretaceous 
Period soon after dinosaurs populated the coastline of the 
retreating Interior Seaway. Most of the rapidly moving, 3- 
min presentation showcases our discoveries with faculty and 
students working at our sites. The penultimate slide is the 
typical PBL entry document, and it prompts, "Obviously this 
was a very different world. How so?" The final slide presents 
marching orders, "You and your peers will work with many 
fossils in your new jobs as professional rangers in Big Bend 
National Park. You will be expected to present inspired 


observations and interpretations of recently discovered 
fossils. The director of the U.S. National Parks and a team 
of scientists will be visiting in three months. They will expect 
a detailed accounting and interpretation of this unique 
resource." 

The video sets the scenario within which investigations 
will occur—in this case, various fossil sites. It also presents 
the expectation that much research into the appropriate 
literature will occur. Some students immediately appreciate 
the benefit of prior knowledge, especially those who have 
had Earth Science, organismal biology, or botany courses. 
Others seem lost at this early point, a condition that rapidly 
lessens as students have the opportunity for discussion and 
fossil analysis in their groups. 

Driving Question 

A driving question is the prompt for students to begin 
their work. Crafting an essential question is not trivial 
(Wiggins and McTighe, 2006). It helps the instructor focus 
the inquiry, and it must both interest and guide the students. 
The question must be open ended, complex, and challeng¬ 
ing. Driving questions typically promote multiple approach¬ 
es to problems (sometimes in several disciplines), all of 
which are subject to numerous detours. This question 
requires authentic artifacts such as research evidence 
presented at a professional meeting. It must be aligned with 
a professor's instructional objectives. Finally, a good driving 
question is provocative. Following much discussion, the 
driving question for AoD in the fall of 2012 was, "How can I 
characterize the Cretaceous ecology of the Big Bend using 
these extinct organisms as my guide?" This driving question, 
or thesis, conveyed a sense of purpose. It was provocative, 
stealthily complex, and, perhaps most importantly, correlat¬ 
ed directly with the goals of this course. 

Perhaps such a question sounds too simple or, at least, 
perfectly straightforward. It was not. The immediate 
response from students could be expressed as, "No big 
deal." Fifteen minutes later, that collective opinion changed 
considerably as the students began to appreciate the expanse 
of a seemingly trivial question. Members of each group were 
digging into the literature. A queue formed. Students 
disappointedly learned that the instructor in a PBL course 
is only one source of knowledge. That instructor is also 
prone to asking additional questions rather than simply 
presenting answers. Using questioning as inquiry is certainly 
considered best practice (Blosser, 1990). As instructors, we 
believe that properly managed chaos can be constructive. 
The first group to respond attempted to work out a concept 
map using Cmap software (Cmap, 2013). This was an 
excellent, and unprompted, approach by the group to 
establish correlations among fossil elements and tenuously 
understood processes (taphonomy, sedimentology, etc.). 
Others groups soon decided that Cmap constructions were 
useful tools for analyzing paleoecological relationships such 
as this tentative one for the Aguja Formation (Fig. 1). 
Concept map reconstructions always vary in completeness 
and accuracy, and all provide numerous seeds for questions. 

Following several minutes of animated discussion in 
their groups, the students produced a lightly coached, 
collective protocol deciding that they must first identify as 
many of the specimens as they could. Next, and to the best 
of their ability, they would determine the ages of key fossils, 
followed by environmental characteristic of most taxa, and 
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FIGURE 1: Student-generated Cmap food web reconstruction of the Javelina Formation. 


then construct a paleoecological scenario that included food 
webs. Students also soon learned that initial identifications 
and assumptions can be wrong and that they must integrate 
all of their "discoveries" into a continuously evolving 
"answer." 

Knows and Need-to-Knows 

Students begin each week by listing what they know 
and what they think they will need to know. Students are 
required to be metacognitive. This process is as fundamental 
as it sounds. For the first cut, students frequently produce a 
list of knows that includes placing new specimens in an 
ever-improving stratigraphic column in the horizon from 
which it was collected and recording whether the fossil was 
discovered in situ. Noting fossil condition (bite marks, 
abrasion, etc.) usually follows. In the need-to-know catego¬ 
ry, identification is usually primary, followed by paleoenvir- 
onment association and other puzzles such as an organism's 
role in a food web. Online research persists throughout the 
semester, as students convert need-to-knows into knows. 
Students effectively explore as many facets as time permits. 

Student Work 

During the discovery of new fossils in the first class of 
the week, each group of four or five runs through the knows 
and need-to-knows, organizes tasks to accomplish, and 
divides the work among themselves. Even though there is a 
self-organized division of labor, each member participates in 


some facet of all of the work. By the end of the period, each 
group must draft a rough outline of a credible presentation 
for the upcoming Thursday. During the following class, we 
see groups working out final details and individuals 
preparing to present their presentations. 

Fossils Presented to the Class 

The fossils (Table I) are almost all research specimens 
from our various research sites. A detailed list is presented to 
demonstrate the diversity and difficulty of the specimens. 
Many are presented with rock matrix either adhering to or in 
a separate bag tied to the specimen. Outcrop B is from the 
uppermost Boquillas Formation, the oldest unit in this 
section. It contains a spectacular selection of giant Platycer- 
amus encrusted (puzzlingly) on both valves with Pseudoperna 
congesta (Montgomery, 2012a). Other fossils include teeth 
and exceedingly rare preserved cartilage from Ptychodus. 
Mosasaurs and turtles are nearby but have not yet been 
incorporated into the collection. 

Pits A to C are located in a small area in the overlying 
Aguja Formation at a site 12 km to the east of Outcrop B. 
These sites have been quarried by Montgomery and 
associates over the past decade. Pit A, the oldest, contains 
only reworked oysters and shark teeth. Pit B is a recent 
discovery that is rich in microvertebrates (Montgomery, 
2012b) and spectacularly preserved leaves above logs with 
Teredolites. At the top of the section. Pit C continues to be 
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TABLE I: Fossils presented to the class. 


Lowest in Section 

Pit A 

Pit B 

Pit C 

Highest in Section 

Outcrop B 

Outcrop J 

Platyceramus platinus 

Crassostrea sp. and 
minor Venericardia sp. 

Lepisosteus sp. scales 
and teeth 

Hadrosaur maxilla, 
teeth, ribs, vertebrae, 
sacrum, centrum with 
tooth marks, femur, 
tibia, fibula, long bone 
fragments 

Pterosaur long bone 
fragments 

Psuedopema congesta 

Scapanorhynchus texanus 

Hybodus sp. 

Chasmosaur cranial 
elements, femur 

Tyrannosaurid tooth 

Ptychodus mortoni teeth 
and preserved cartilage 


Lissodus sp. 

Ornithomimid longbone 

Theropod coprolite 
containing bone 



Ptychotrygon sp. 

Juvenile theropod neural 
spine 

Unidentified broken and 
fused ribs 



Ischyrbiza sp. 

Ossified tendon 
fragments 

Adult Alamosaurus tooth, 
cervical vertebra, rib 
fragments, ischium, tibia, 
femur 



Serpentes sp. 

Soft-shelled turtle 
fragments 

Juvenile Alamosaurus 
vertebra 



Albanerpeton sp. 

Unidentified rib and 
longbone fragments 

Lepisosteus scales 



Scapherpeton sp. 

Dinosaurian carnivore 
and herbivore coprolites 

Soft-shelled turtle rib 
fragments 



Alphadon sp. and 
Cimolodon sp. teeth 

Woody vines, leaves, 
stems, logs, amber, 

Sabal bigbendense seeds 

Unio sp. 



Probable pterosaur, bat, 
frog, and other skeletal 
fragments 

Shell (Continuoolithus) 

Viviparus sp. 



Spheroolithus 


Gyrogonites 



Teredolites and log 
remnants 


Crayfish burrows 



Sabal palm seeds 


Palm wood 



Well-preserved leaves 




expanded as it regularly produces abundant and sometime 
unique skeletal material, mostly from dinosaurs. 

At the top of the local Cretaceous section. Outcrop J is in 
the overlying Javelina Formation and is located 12 km to the 
northeast. This area displays a rich Maastrichtian lacustrine 
landscape complete with a diverse fauna and was the subject 
of recently published work (Montgomery and Barnes, 2012). 

Our field team collected all of the fossils. We are, thus, 
familiar with each specimen in context. This familiarity 
greatly facilitates answering student questions, or, more 
precisely, coaching their investigations. As is common for 
fossils in the Aguja and Javelina Formations of West Texas, 
most are fragmentary. Some of the fossils have definitely 
been transported and are not coeval (the palm wood, for 
example). A few skeletal elements show strong evidence of 
predation such as pits and long scars probably rendered by 
crocodiles or theropods. The microvertebrates in Pit B 
require stereomicroscopes and steady hands for study. 
Provisions were made throughout the course for students 
to photograph all of the fossils, including the microverte¬ 
brates. The images facilitate their description in notes and 
discussion in presentations. 


Student Assessment 

Measurement of student learning outcomes is framed by 
authentic procedures that seek evidence of active, compre¬ 
hension learning on real-world, intellectual tasks without 
simple recitation of memorized information. Much research 
and careful analysis is required in this course. Despite careful 
work, some fossil identifications remain uncertain, and the 
ecological roles of extinct animals are difficult to define. 
Student output includes weekly electronic notebooks re¬ 
views, brief weekly and detailed final presentations, and 
evaluation of the final report—an extensive, original, and 
creative product. Students work in small groups, but all 
assignments are individual and are graded using a rubric 
(Table II) with which the students are familiar. A rubric for 
each student for each week is maintained. The point total for 
that student for each week begins with the first discussion 
each Tuesday and is not completed until after the final 
presentation. Thus, a student's grade for the week is an 
ongoing assessment not only composed of notebook and 
presentation but also influenced by instructor impressions of 
individual work noted while roving among the groups, in 
meetings during office hours, and via electronic media. 
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TABLE II: Rubric used by instructors and students to assess assignments. 



Exemplary (3 pts.) 

Proficient (2 pts.) 

Ineffectual (1 pt.) 

Incomplete (0 pts.) 

Points 

Research & Critical Thinking 

Content 

Accurate presentation 
of all characteristics of 
all specimens. 

Accurate presentation 
of most characteristics 
of all specimens. 

Inaccurate presentation 
of characteristics of a 
few specimens. 

Inaccurate presentation 
of characteristics of 
several specimens. 


Research 

Rich variety of peer- 
reviewed sources 
correctly evaluated and 
synthesized. 

Required minimum of 
sources, correctly 
evaluated and 
synthesized. 

Minimal research 
lacking support for 
arguments. 

No appropriate peer- 
reviewed sources and/ 
or incorrect 
interpretations. 


Synthesis 

Inspired ideas and 
clearly based on 
research. 

Research-based and 
integrates ideas in a 
cohesive product. 

Researched, but 
incompletely integrates 
ideas in a cohesive 
product. 

Series of unlinked 
concepts and/or work 
may not be logical 
and/or verifiable. 


C ommunication 

Oral 

Logical presentation 
with rich detail that 
indicates synthesis of 
concepts. 

Logical presentation 
that audience can 
easily follow. 

Presentation with a 
few flaws in logic, but 
audience mostly 
follows. 

Flaws in logic and 
lacks sufficient detail 
for the audience to 
understand content. 


Written 

Indicates higher level 
thinking with accurate 
observations, 
appropriate references, 
and inspired synthesis. 

Accurate observations 
and appropriate 
references. Reasonable 
syn thesis. 

Mostly accurate 
observations and 
appropriate references, 
but some flaws, 
especially in synthesis. 

Mostly inaccurate 
observations and 
lacking appropriate 
references. Poor to 
missing synthesis. 


Graphics 

All excellent (size and 
color) and support 
content. 

Suitable, but not 
inspired, and all 
support content. 

Mostly serviceable, but 
not all support content. 

Not fitting and poorly 
support content. 


Creativity 

Creativity 

Demonstrates spectrum 
of original thought. 

Ideas are inspired and 
ingenious. 

Some original thought. 
Several new ideas and 
insights. 

Several original (or, at 
least, logical) ideas and 
insights. Little evidence 
of original thinking. 

Little to no evidence of 
original thinking. 


Collaboration 

Teamwork 

Clear evidence of 
continuous and mutual 
effort in the creation of 
final product. 

Evidence of working 
well together, but 
lacking some evidence 
of integrated 
teamwork. 

Some problems 
working together. 
Lacking collaboration. 

Not a collaborative 
effort. 


Score for the week (adjusted maximum of 10.0) 



including text queries. (Rubrics for each student for each 
week are now kept on an iPad). All notebooks receive 
extensive feedback. (An interrater reliability process is 
currently being implemented.) Opportunities to rethink 
and rewrite are standard procedure. Figuring into the 
instructor's evaluation are assessments carried out by each 
group member evaluating the performance of every other 
member of that group. The rubric used for this process is 
peer Collaboration and Teamwork (Buck Institute for 
Education, 2013). 

IMPLEMENTATION AND EVALUATION 

Delivery 

Despite working on the same fossils, each group has 
different ideas concerning interpretation, even if most 
students eventually arrive at the same taxonomic identifica¬ 
tions. Roving instructors (professors and teaching assistants, 
when available) independently vet these interpretations each 
week for each group. Thus, there is not a lot of sharing 


among groups on Tuesdays, but there is a great deal of 
discussion among groups on Thursdays. Instructors find that 
some groups are more adept at the use of technology-based 
resources than are others. In these cases, we might refer the 
group to Cmap or another such resource. An air of urgency 
to present discoveries and analysis, perhaps driven by a spirit 
of gentle competition, appears to be constructive in this 
situation. 

Beginning in the first class, students quickly come to the 
correct assumption that there are no step-by-step instruc¬ 
tions to formulating and answering their questions. Perhaps 
the major difference between what students (probably) did 
in the usual science class and what they do in this class is 
that in a PBL class, students construct their own solutions to 
questions that they generate. These are quite intelligent 
students, but rarely have they had to ask the questions. They 
are experts at providing (frequently memorized) "correct" 
answers to questions that they expect to see on multiple- 
choice or short-answer tests. For scientists, correct answers 
can be elusive. An early correct answer may be proved 
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incorrect later. In paleontology, for example, students learn 
that a possible, or even probable, correct answer may exist 
within a range of possible taxa (within a range of possible 
correct answers). Or it may not. PBL is rather remote from 
our usual standardized and predictable educational upbring¬ 
ing. The experience can be unsettling for students who are 
practiced in a more traditional learning environment. 

Presentations must be delivered in a modified Pecha- 
Kucha (PechaKucha, 2013) format with 10 slides timed for 
15 s each (10x15) rather than the specified 20x20. The 
format was designed by architects Astrid Klein and Mark 
Dytham to prevent the usual talk and/or PowerPoint 
presentation from going on and on. PechaKucha has gone 
viral and is used worldwide for creative presentations. In our 
version of this format, only images are presented. No charts, 
tables, or bulleted text is permitted. Each image should 
suggest a concept, and each speaker must thoroughly 
understand the material because there are none of the usual 
bulleted prompts. One benefit is that no student takes more 
than the allotted time. Following each presentation, addi¬ 
tional time is provided. An unexpected and welcomed 
outcome of using PechaKucha is the surprisingly numerous 
and perceptive questions that each presentation invites. We 
consider this inquiry-driven presentation methodology to be 
a major improvement on the usual process. 

Experiences Implementing the Course 

We found that a great deal of preparation was required 
to assemble a diverse collection of fossils that would allow a 
rich reconstruction of the area during a time of changing 
climate, changing sea level, changing fauna and flora, and 
ending with the extinction of nonavian dinosaurs. Students 
in our master's program for science teachers also report that 
preparation for a new PBL course is substantial. 

Instructors must learn to carefully listen and must not be 
quick with answers. We provide clues and/or resources 
rather than answers. Simply rephrasing a question will 
frequently lead to another student in the group having an 
epiphany. Following these revelations, students explain 
discoveries to other students. If another group runs into 
the same question, we ask the student who had the 
epiphany to visit that group and to help its members solve 
their problem (without giving the answer). We were 
surprised how rapidly the students picked up this technique. 

The daily use of a rubric with which the students are 
familiar will continue. Conducting evaluations during each 
class is crucial. We did not experience any students falling 
behind as is common in some traditional classes. Students 
expect us to meet with them several times during each class. 
We expect the students to make careful observations and to 
conduct detailed research. We expect results. So do they. 

Evaluation of Course Design and Delivery 

Survey results prompted modifications in course design. 
Among the changes are more examples of successful 
previous projects and examples of how those projects 
improved during the course of the semester. Pursuant to 
this goal, we preserved examples of student work that will be 
posted on eLearning. The detailed and ever-evolving 
notebook format will remain. The PechaKucha presentations 
remain as is. Questions posed by other students are 
perceptive, learned, and provocative, quite unlike anything 
we have experienced in our traditional classes. Certainly, a 


major contributor to this enthusiasm is not in the format but 
in having the students primed to question in this probing 
manner by having practiced the technique during each class. 
PechaKucha simply provides an excellent platform. 

Student Performance Evaluations and Survey Results 

Evaluation was accomplished in four ways: question¬ 
naire data from an instructor-designed instrument, univer¬ 
sity questionnaire results, free response statements, and 
weekly grade data throughout the semester. There is no 
other course to use for comparison. The previous version of 
this course was conducted in a lecture/laboratory format 
with nonhonors students wherein memory was stressed 
more than problem solving. 

Student Performance 

Student performance as assessed using the course rubric 
(Table II) began the second week and showed week-to-week 
variation over the semester. Out of 10 points, scores for 
week 2 through week 13 were 8.0, 9.6, 9.0, 8.8, 8.6, 8.6, 9.4, 
9.0, 9.4, 9.4, 8.8, and 9.8. The trend in these scores is subtle, 
but it is certainly noticeable in class. Students adopt best 
practice and become comfortable with the class about 
halfway through. These numerical results reflect an interplay 
of regular improvement in student performance, in addition 
to varying difficulty of sample identification and interpreta¬ 
tion and heightening of expectations for student research. 
Student course objectives 1 and 2, "Students will be familiar 
with evolutionary history of numerous Mesozoic fossils" and 
"Students will have a basic understanding of Mesozoic 
geology and paleoecology," respectively, were assessed 
weekly with the Content section of the rubric (Table II), as 
well as through evaluation of the final projects. In addition, 
the fulfillment of course objective 5, "Students will produce 
an original research project of extraordinary quality," was 
evident in the diverse and skillfully executed PechaKucha 
presentations discussed earlier. For details of a project of 
"extraordinary quality," see the Oral, Written, Graphics, and 
Creativity sections of the project rubric (Table II). Final 
projects were all highly coached through questioning, 
providing research, and showing examples. All finals 
projects met or exceeded the expectation of the instructors. 
Our long experience with honors students suggests that we 
can expect similar, highly satisfactory results in future 
semesters. 

Questionnaire Data 

Statements in the postcourse survey (Table III) were 
composed before the class began to help guide our 
instructional philosophy. Regular contact with the students 
certainly provided nearly continuous formative assessment 
that resulted in numerous minor modifications by the 
instructors during the semester, such as adjusting difficulty 
of fossils or modifying expectations for student research. 
Two weeks after grades were posted, all students were 
emailed the survey to assess their opinions about this class. 
This wait period before surveying was to hopefully provide 
time so that students would give us an overall view of the 
course. Survey responses went directly to the department 
secretary, who removed all identifiers. Eight students 
returned surveys, while 10 completed the university evalu¬ 
ation. The instrument was a standard 5-point Likert scale 
survey with a neutral choice having a value of 3. Statement 
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TABLE III: Likert questionnaire data. 1 



Statement 

Means 

1 

The class helped me learn to obtain information from a variety of sources. 

4.5 

2 

"Discovering" new fossils each week approximated the work of paleontologists. 

4.5 

3 

I honed my skills for working in groups. 

4.0 

4 

My research skills have improved over the course of the semester. 

4.5 

5 

The research document that I completed is of substantial merit. 

4.4 

6 

This course helped me to evaluate new information and to reassess my knowledge. 

4.8 

7 

I gained confidence in my understanding of how to properly conduct scientific inquiry. 

4.4 

8 

If given an opportunity, I would like to take another PBL class. 

4.6 

9 

Scientific inquiry requires consulting peer-reviewed research. 

4.8 

10 

I gained an understanding of basic principles and concepts of geology and paleontology. 

5.0 

11 

The next time I go to a museum, fossils will appear much more understandable. 

5.0 

12 

I can apply the general principles that I learned in this class to other topics. 

4.4 

13 

I grew in my ability to argue a point. 

4.1 

14 

Students of every major should experience scientific inquiry in at least one course. 

4.6 

15 

Knowing little to nothing about various fossils during the first weeks made me uncomfortable. 

3.8 


1 Point values: 1 = strongly disagree; 2 = disagree; 3 = neutral; 4 = agree; 5 = strongly agree. 


15 was analyzed separately, as responses registering the 
greatest agreement with a negative statement have the 
highest value. 

Survey Composed by Instructors 

Likert responses are probably best treated as ordinal 
data because there is no statistical certainty whether the 
intervals between values are equal. For this dataset, modes 
for each statement have a value of 5 with the exception of 
Statement 3, which had a value of 4. As is common practice, 
we can determine means, but we will make no claims based 
on intervals such as "twice the value" when comparing a 
"disagree" (value of 2) to an "agree" (value of 4). Means 
present a more valuable analysis tool as long as one observes 
the caution concerning comparing intervals. 

Statement 15 (Table III) suggests some uneasiness 
existed at the first of the semester as a result of unfamiliarity 
with fossils. Average responses for the other statements are 
all in the "agree" to "strongly agree" range, indicating an 
overall positive response by students to these goals for the 
course. Two "disagree" outliers are present, with one to 
Statement 5 and one to Statement 13. 

The survey data with mean values of 4.0 to 5.0 ("agree" 
to "strongly agree") present an overall student perception of 
accomplishment in the categories selected for this question¬ 
naire. Students studied specimens, engaged in research, 
formulated models, presented findings, and argued impor¬ 
tant points. They reported that they would like to take 
another PBL class (4.6). Statement 15 was interesting in that 
students reported with mild agreement (3.8) that encoun¬ 
tering unknowns made them uncomfortable. They rapidly 
progressed beyond this state of mind. We will add a prompt 
next semester seeking to further qualify and quantify change 
in students' comfort levels by the end of the semester. 

Free Responses 

The free response selections reflect both the most 
common themes and the most extreme. 


Strengths of the course were mostly along the lines 
reported by this student: 

"The availability of hands-on materials connected the 
information gained through occasional lectures with real 
work performed by paleontologists. I think many students 
create separate and disconnected spheres in their minds for 
'classroom learning' (lecture-based, little to no collaboration 
with other students) and 'out-of-class/real world/occupa¬ 
tional learning' (experience-based, mix of solo research and 
group work), and I think the PBL format helps to bridge this 
gap. It's not that lecture-based courses are never useful or 
effective, in my opinion, but there is a tremendous bias 
towards them at educational institutions even though they do 
not represent what students will be doing for the rest of their 
lives. Many occupations that call for higher education 
require working with other individuals as part of a team to 
solve problems or accomplish tasks, yet there are few 
opportunities for students to experience that atmosphere in 
college." 

Another expression of enthusiasm for the PBL method 
came from this student: 

"When everyone thinks alike, no one is really thinking. In 
this PBL format, everyone contributes different ideas to the 
discussions, thus stimulating creativity and an eagerness 
from each student to prove the correctness of his or her 
beliefs." 

Weaknesses reported for the course mostly reflect what 
we hoped would be start-up issues that would not be 
overlooked the next time the course was taught in the spring 
of 2014. This quote was not uncommon in that students did 
not respond favorably to lectures that by definition could not 
be focused on the topic each group was investigating during 
that part of class. Lectures lasted approximately 4-20 min. By 
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TABLE IV: Likert questionnaire data from University of Texas at Dallas course survey. 1 


Prompt 

Means 

The course objectives were clearly defined. 

4.30 

The course was well organized. 

4.70 

Overall, the course was excellent. 

4.78 

The instructor was well prepared in the subject area. 

4.80 

The instructor communicated information effectively. 

4.50 

The instructor seemed genuinely interested in teaching. 

4.80 

The instructor evaluated students fairly. 

4.80 

Overall, this instructor was excellent. 

4.80 

I was free to ask questions and express my opinions and ideas. 

4.70 

My performance was evaluated fairly. 

4.60 

I discussed ideas from this course with others outside the classroom. 

4.50 

This course has been (or will be) of value to me. 

4.30 

This course inspired me to learn more. 

4.50 


'Point values: 1 = strongly disagree; 2 = disagree; 3 = neutral; 4 = agree; 5 = strongly agree. 


the end of the semester, lectures were pared to approxi¬ 
mately 5 min. 

"The lectures are beneficial, but I believe that these should 
supplement a textbook reading, which the student can read 
on his or her own." 

But then, others replied like this: 

"The occasional lectures are an important part of the 
students' knowledge, and the materials facilitate scientific 
inquiry. ” 

Some students experienced pronounced frustration 
when given fossils that even professionals could not identify 
other than being "from a small, herbivorous dinosaur, 
probably." We sought to give students an authentic 
experience, and there are simply some fossils such as a 
small piece of a rib that leave all of us scratching our heads 
about any specific taxonomic assignment. Some students 
were happy to get close to the answer and then move on. 
Others simply could not let go without making heroic 
attempts to figure out every possible detail. The idea that a 
fossil might remain poorly identified was anathema to them. 
Grappling with problems that have limited solutions was a 
novel experience. We considered removing the few fossils in 
this category (five or six specimens) but decided to keep 
them in the course collection. The instructor plays a pivotal 
role in balancing frustration and learning. This is only 
accomplished via regular discussions with each student. We 
agree with this student's statement, but we prefer to wait to 
offer help until a student's best efforts have been exhausted: 

"It was very aggravating when coming across the fossils and 
not being able to figure out the identity after long 
examination and research. Perhaps more information about 
the fossils could be given to the students, rather than leave 
them 'in the dark' on many fossils that even professional 
paleontologists have trouble identifying." 


These student responses make it clear that student 
objective 3, "Students will evaluate fossils from several of our 
dinosaur sites in west Texas using criteria presented in the 
class and in primary sources" was met. In addition, objective 
4, "Students will apply their learning and understanding to 
real-world research projects," is evident in the student 
responses given here. Even the frustrations of the students 
mimic those of scientists in the field. Both of these objectives 
were also assessed weekly in the Research, Synthesis, and 
Teamwork sections of the rubric (Table II). 

Questionnaire Data From University of Texas at Dallas 
Course Survey 

Student responses to the university course evaluation 
instrument were positive, with 9 or 10 students responding 
to each prompt. Statements that go to the theme of this 
investigation are listed in Table IV. These scores would seem 
to exhibit student satisfaction, particularly for the first time 
such a different methodology is employed and for the first 
time any student had taken a PBL course. 

Free response statements on the course survey (Table V) 
provide both reassurance and pause for thought about 
possible modifications. In the need improvement category. 
Statements 3 and 6 reflect uncertainty about assignments 
that were not made clear by Montgomery at the beginning of 
the semester. Modifications based on Statements 7-9 will be 
incorporated in the next offering of this course. 

SUGGESTIONS AND CONCLUSION 

Questionnaire data indicate a high level of student 
satisfaction with the course and with the process, in addition 
to successfully meeting the class objectives. Upon occasional 
quizzing during the course, students reported that the 
workload, while sometimes "substantial," was not onerous. 
Another anecdote of note is that the biology students were 
particularly pleased not to be required to memorize 
voluminous amounts of information that required neither 
active learning nor creative construction of research-based 
solutions. While presenting a PBL course can involve a 
daunting amount of upfront work for the instructors, this 
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TABLE V: Free response statements on the University of Texas at Dallas course survey. 


Prompt 

Statements by Students 

What aspects of 
this course should 
remain the same? 

1. Everything This class is very fun and dynamic. 

2. Class size is excellent, as is the class format. The hands-on, discussion-based class is a welcome break from 
the large, lecture-based classes that are more common. REAL FOSSILS AND MICROSCOPES 

3. Method of instruction 

4. The process at which we look at and evaluate fossils 

5. The hands-on approach and being able to work individually in groups to solve problems. 

6. The hands-on structure of the course was excellent and needs to be not only kept, but emulated in other 
classes. 

What aspects of 
this course need 
improvement? 

1. There were times when things were a bit poorly organized... but for the most part the professor was good 
about explaining everything in class. I imagine it would have been VERY difficult if I had had to skip any 
classes though. 

2. Friday night deadline for weekly assignments was a bit inconvenient at times. Perhaps a Sunday night 
deadline would work better for students, but I dont know if it would make grading more difficult for the 
instructor. 

3. Telling the students to make a cumulative report what is needed from the start would help, but was a small 
inconvenience. 

4. No changes needed. 

5. Objectives and requirements for the assignments should be clearly defined early in the semester. 

6. A little more basic information about paleontology at the beginning of the semester would be nice. 

7. Greater focus on learning about specific dinosaurs in class. Provide in depth background information on 
dinosaur origins or time frame. 

8. A little more instruction—maybe a bit more guidance in the beginning on how to identify fossils what are 
good bad sources to use, so that later in the course students can get the right information on the more 
complicated subjects. 

Additional 

comments: 

1. Wonderful CV class. Really pushes students. A must-take. 

2. Favorite class I have ever taken I have learned a great deal without being too overwhelmed by the subject. I 
would recommend this class to anyone. 

3. I thoroughly enjoyed this class 

4. Excellent course 

5. It was a very fun class. 


load lessens for subsequent classes. By no means should one 
conclude that a PBL course would become static, however. 
We continue to include numerous new discoveries in the 
fossil rotation. As our understanding of these puzzling sites 
grows, the course will clearly change to reflect this 
understanding. The students are certainly aware of the 
somewhat fluid nature of the discovery process. We propose 
that students' recognition of knowledge building rather than 
simple learning is a powerful lesson (after the ideas of 
Scardamalia and Bereiter, 2007). 

Implementing a PBL course requires both an under¬ 
standing of the methodology and a fundamental relinquish¬ 
ing of the role of instructor as purveyor of all knowledge. 
Students gain the responsibility for constructing their own 
knowledge via a process not unlike that in which their 
researcher/instructor engages. In this example with honors 
students, undergraduates were sufficiently mature both to 
drive their own learning and to do this in concert with their 
peers. Such a process mimics the modern workplace 
wherein experts work in groups, gather data, solve 
problems, and present well-reasoned solutions. We would 
argue that PBL not only is an effective learning methodology 
but also is excellent preparation for the future careers of 
these inquisitive and industrious students. 
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